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ABSTRACT 

The Incentives for Improvement Program is an 
alternative student evaluation and recognition system that is 
responsive (all students have a realistic chance to achieve success) 
and challenging (students are not likely to succeed consistently 
unless they work up to their potential). The program's goals are to 
raise student performance and foster students' motivation to learn. 
An evaluation was conducted to determine whether the program 
accomplished its goals during its first year of implementation. 
Volunteer teachers from four Baltimore City (Maryland) middle schools 
participated in the program during the 1989-90 school year. The 
program's effectiveness in raising students' grades, probability of 
passing, intrinsic interest in their schoolwork, effort, and 
self-concept of ability was evaluated by comparing end-of- school-year 
outcomes for students in participating classes with those of similar 
students who were enrolled in the same courses at four other 
Baltimore City middle schools. To make these comparisons as precise 
as possible, pre-test adjusted outcome measures in hierarchical 
linear models were used. The results illustrate the substantial 
positive impact of individualized, improvement-oriented reward and 
recognition structures on students' grades iii participating courses 
and .on their probability of passing these coursea. There was also a 
small positive effect of the program on students' self -reported 
levels of effort. The program provides an evaluative process in which 
educationally disadvantaged students share increased opportunities to 
experience success in a challenging curriculum by earning recognition 
for academic improvement and by building upon this improvement to 
earn better grades and higher passing rates. Teachers' expectations 
that students will succeed academically are a vital part of 
motivating and effectively teaching current;? low-achieving students, 
included are 25 references, 3 tables, 4 figures, and an appendix 
providing selected questionnaire items used to measure students' 
perceptions. (Author/RLC) 
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The Center 



The mission of the Center for Research on Effective Schooling for Disadvantaged Students 
(CDS) is to significantly improve the education of disadvantaged students at each level of 
schooling through new knowledge and practices produced by thorough scientific study and 
evaluation. The Center conducts its research in four program areas: The Early and Elementary 
Education Program, The Middle Grades and High Schools Program, the Language Minority 
Program, and the School, Family, and Community Connections Program. 

The Early and Elementary Education Program 

This program is working to develop, evaluate, and disseminate instructional programs 
capable of bringing disadvantaged students to high levels of achievement, particularly in the 
fundamental areas of reading, writing, and mathematics. The goal is to expand the range of 
effective alternatives which schools may use under Chapter 1 and other compensatory education 
funding and to study issues of direct relevance to federal, state, and local policy on education of 
disadvantaged students. 

The Middle Grades and High Schools Program 

This program is conducting research syntheses, survey analyses, and field studies in middle 
and high schools. The three types of projects move from basic research to useful practice. 
Syntheses compile and analyze existing knowledge about effective education of disadvantaged 
students. Survey analyses identify and describe current programs, practices, and trends in middle 
and high schools, and allow studies of their effects. Field studies are conducted in collaboration 
with school staffs to develop and evaluate effective programs and practices. 

The Language Minority Program 

This program represents a collaborative effort The University of California at Santa 
Barbara is focusing on the education of Mexican- American students in California and Texas; 
studies of dropout among children of recent immigrants are being conducted in San Diego ard 
Miami by Johns Hopkins, and evaluations of learning strategies in schools serving Navajo 
Indians are being conducted by the University of Northern Arizona, The goal of the program is 
to identify, develop, and evaluate effective programs for disadvantaged Hispanic, American 
Indian, Southeast Asian, and other language minority children. 

The School, Family, and Community Connections Program 

This program is focusing on the key connections between schools and families and between 
schools and communities to build better educational programs for disadvantaged children and 
youth. Initial work is seeking to provide a research base concerning the most effective ways for 
schools to interact with and assist parents of disadvantaged students and interact with the 
community to produce effective community involvement. 



Abstract 



The Incentives for Improvement Program is an alternative student evaluation and recognition 
system that is both responsive (all students have a realistic chance to achieve success) and 
challenging (none are likely to succeed consistently unless they work up to their potential). The 
program's goals are to raise student performance and to foster students' motivation to learn. An 
evaluation study was conducted to determine whether the program had accomplished these goals 
during its first year of implementation. Volunteer teachers from four Baltimore City middle 
schools participated in the program during the 1989-90 school year. The program's effectiveness 
in raising students' grades, probability of passing, intrinsic interest in their schoolwork, effort, and 
self-concept of ability was evaluated by comparing end-of-school-year outcomes for students in 
participating classes with those of similar students who were enrolled in the same courses at four 
other Baltimore City middle schools. To make these comparisons as precise as possible, pre-test 
adjusted outcome measures in hierarchical linear models were used. The results indicate that the 
program had substantial positive effects on students' grades in participating courses and on their 
probability of passing these courses. There was also a small positive effect of the program on 
students' self -reported levels of effort. 
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Introduction 



Teachers provide students with feedback on 
their performance by assigning grades and 
making comments on students' quizzes, 
tests, and major assignments. They often 
give special recognition or awards to 
students who do well. Although feedback, 
evaluation, and recognition practices are 
supposed to help motivate students tc reach 
higher standards of intellectual achievement, 
teacher reports concerning the proportion of 
students who are not performing up to 
potential suggest that typical current practices 
are not very effective. 

There have been decades of research on 
student and worker motivation. The 
principles of goal-setting theory derived from 
this research suggest specific, practical 
alterations that teachers can make to their 
feedback, evaluation, and recognition 
practices that will make them more effective 
in motivating students to work hard at 
learning activities. 

Volunteer teachers from four Baltimore City 
middle schools agreed to make these 
alterations by implementing the Incentives for 
Improvement Program - a system in which 
feedback and recognition is based upon 
evaluating students in reference to specific 
improvement goals; individualized, short- 
range performance benchmarks that are 
challenging but reachable. This paper 
evaluates the effectiveness of this program in 
its first year of implementation. 

One basic premise of goal-setting theory is 
that task performance is heavily influenced by 
the conscious goals that individuals are trying 
to accomplish on the task (Locke & Latham, 
1984). Specific challenging goals (e.g. "beat 
your highest previous score") lead to better 
performance than specific easy or vague 
goals (e.g., "do your best") or no goals 
(Locke fcLatham, 1990b). But, in order for 
goals to lead to better performance, students 
must be consciously trying to attain those 
goals (Erez & Zidon, 1984) and must receive 
feedback which allows them to judge whether 



they are attaining the goal (Becker, 1978; 
Strang, Lawrence, & Fowler, 1978). 

Goal commitment is highest when goals are 
perceived as reachable rather than impossible 
and when there are clear payoffs (intrinsic or 
extrinsic) associated with attaining the goal 
(Locke, Latham, & Erez, 1988). Also, 
assigned goais that represent an appropriate 
level of challenge typically produce the same 
level of goal commitment and performance as 
participatively et or self-selected goals 
(Latham & Let- 1986; Locke & Latham, 
1990a). 

Traditional feedback, grading, and 
recognition practices in schools are based on 
evaluation systems that compare students' 
performance to that of other students or to 
desirable absolute standards of achievement: 
The implicit goal assigned to students in both 
types of systems is to attain desirable grades 
and the recognitions such as honor roll 
membership that go along with such grades. 

Academically disadvantaged students ~ who 
by the middle school years are significantly 
behind more advantaged classmates in 
academic skills - may Find it impossible to 
attain this implicit goal even if they work very 
hard. Even dramatic progress can still leave 
disadvantaged students near the bottom of 
their class in comparative terms and far from 
the absolute levels of performance that are 
rewarded and recognized in their school. 
Once students begin to realize that their best 
efforts will go unrecognized and unrewarded, 
they become frustrated with and disengaged 
from school (Natriello, 1982), and their goal 
commitment, level of effort, and rate of 
progress drop precipitously. 

One reason traditional evaluation practices are 
ineffective is that they include neither specific 
improvement goals nor regular improvement- 
focused feedback to make students 
accountable for making consistent, gradual 
progress toward high levels of achievement 
and understanding. Traditional practices do 
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not explicitly encourage all students 
(regardless of current class standing and 
achievement level) to try to raise their 
performance levels by specific amounts in 
specific time periods. 

In the Incentives for Improvement Program, 
students are assigned specific, short-range, 
individualized goals. The goals are specific 
because such goals are better motivators than 
general goals, largely because progress 



toward a specific goal is easier for students to 
detect. Similarly, the goals are close at hand 
(e.g., "to score 10 points higher on the next 
quiz") because such goals are more 
motivating than distant goals (e.g., "to 
someday become an 'A' student") Finally, 
the goals are individualized, because a 
specific goal that is challenging but doable for 
one student may be unreachable (at least, in 
the short-run) for another student and may be 
too easy for still another student 



Program Components 



The Incentives for Improvement Program 
consists of three major components: (a) for 
each quiz or test, students are given an 
individualized, specific goal at which to aim; 
as students improve, their individualized 
goals are gradually raised; (b) "improvement 
points" are used in scoring quizzes and tests 
(these points provide students with clear 
feedback concerning their success at 
accomplishing the individualized goals), and 
(c) all students who raise their performance 
level receive official recognition through 
various types of improvement awards. 

The Incentives for Improvement Program 
features three-quiz "rounds." At the start of 
each round, each student receives his or her 
"base score" for that round. This base score 



represents the student's average percent 
correct on recent quizzes and provides a 
starting point for improvement. Students are 
asked to try to beat this score on the next 
three quizzes. After each quiz, each student's 
quiz score is compared to his or her base 
score and students earn 0, 10, 20 or 30 
improvement points on the quiz, depending 
on by how much they were able to beat their 
base score. 

Before introducing the program to students, 
teachers give two or more quizzes, then 
determine an initial base score for each 
student by averaging his or her scores on 
those quizzes. Then, on the next three 
quizzes, students are able to earn 
improvement points as shown below: 



Quiz Score Improvement Pts. 

5 or more points below base score 0 

4 points below to 4 points above base score 10 

5 points to 9 points above base score 20 
More than 9 points above base score 30 
95% - 99% (if students' base score is above 90%) 20 
100% 30 



Thus, improvement points are given in 
relationship to past performance. A student 
whose base score (average percent correct on 
recent quizzes) is 65 and who gets a 70 earns 
the same number of improvement points (20) 
as a student whose base score is 75 and who 
gets an 80. In order to earn the maximum 
number of improvement points, a student 
must beat his or her base score by more than 
9 points. However, there is no danger of 
students "topping out" with too high a base 
score; students who have a base score over 
90% receive 20 improvement points when 
they score 95-99%, and receive 30 
improvement points if they get a perfect 
paper. Figure 1 shows how improvement 
points would be computed for one fictitious 
set of students. 



Insert Figure 1 



Figuring Average Improvement 
Points and New Base Scores 

At the end of each round of three quizzes, 
each student's average improvement points 
for the round is computed (See Figure 2). 
Each student's new Base score is then 
figured by averaging the student's percent 
correct on the last three quizzes with the 
current base score. 



Insert Figure 2 



End-of-Round Awards. Students receive 
awards at the end of every round. When the 
program began, two types of awards were 
offered ("Rising Star" and "Milestone" 
Awards). Late in Year !, a few of the 
participating teachers began offering a third 
type of award ("90% Club Awards"). Use of 
this award spread to all schools in Year 2. 
Also, at the beginning of Year 2, Milestone 
awards were replaced by "Personal Best" 
awards. 



Rising Star Awards. A Rising Star 
Award is given to each student who averages 
at least 20 improvement points on the three 
quizzes in a round. Figure 2 shows the 
performance of 15 fictitious students during 
Round 1. Nine of these students earn a 
Rising Star Award for Round 1 because their 
improvement point average (as listed in the 
final column of the quiz score sheet) is 20 or 
greater. 

Milestone and Personal Best Awards. 

Teachers keep a record of students' base 
scores throughout the year (see Figure 3). In 
Year I, students received a Milestone Award 
whenever they raised their base score 5. 
points beyond their initial base score (e.g., a 
student whose initial base score was 65 
received a certificate of recognition if he or 
she reached a base of 70, received a different 
certificate later for reaching a base of 75, and 
so on. 



Insert Figure 3 



In the current program, whenever a student 
reaches a base score that represents a new 
high for him or her - even if it is only one 
point higher — the student receives a 
"Personal Best" award for breaking his or her 
past "personal record." 

90% Club Awards. At the end of every 
round, "90% Club" awards are also now 
given to every student whose new base score 
at the end of the round is 90% or above. 

In summary, at the end of every three 
quizzes, students have the opportunity to earn 
three awards. They receive Rising Star 
awards for averaging at least 20 improvement 
points on the last three quizzes, they receive 
Personal Best awards for setting personal 
"base score" records, and they receive 90% 
Club awards for reaching and maintaining a 
performance level of 90% or better. 



Description of the Evaluation Study 



The purpose of the evaluation study was to 
determine whether the Incentives for 
Improvement Program accomplished its two 
goals: to raise student performance and to 
foster students' motivation to learn. A 
matched control group, pre-test-posttest 
design (Fitz-Gibbon & Morris, 1987) was 
used to evaluate the program's effectiveness 
in reaching these goals. 

Volunteer teachers from four Baltimore City 
middle schools participated in the program 
during the 1989-1990 school year. When 
registering a class for the program, teachers 
were asked to describe the title of the course, 
and the ability levels and grade levels of the 
students. This information was used to 
match participating classes with control group 
classes drawn from four nonparticipating 
middle schools. The nonparticipating 
schools were selected because they had 
student populations that were similar to those 
in the participating schools, according to 
principals' reports. 

Measures 

Intrinsic Value of the Subject Matter, 
Effort, and Self-Concept of Ability. 
Students in each of the participating and 
control classes answered questionnaire items 
that focused on their perceptions of intrinsic 
value (e.g., "How excited are you to learn 
about this subject matter?"), effort (e.g., 
"How hard are you working to learn about 
this subject?"), and self-concept of ability 
(e.g., "How good are you in this subject?"). 
Items had a response scale (ranging from 1 to 
7 unless otherwise noted) with verbal 
anchors at each end of the scale (see 
Appendix). 

Each construct was measured using multiple 
indicators taken from the Motivation to Learn 
Scale (Mac Iver, Stipek, & Daniels, 1991). 
A student's responses to multiple indicators 
of the same construct were averaged to create 
a composite score for that construct These 
measures were collected twice in 22 matched 



pairs of classes; once shortly after the 
participating class was enrolled in the 
program, and once near the end of the fourth 
quarter of the school year (around mid-May). 

Past and Current Performance. 

Information regarding students' grades from 
the year prior to the intervention was 
available only for seventh- and eighth-grade 
students (20 matched pain of classes). For 
these students, a "general average" grade 
across all classes and all reporting periods 
from the pre- intervention year was computed 
and used as a pre-test measure. Each 
student's performance in a participating class 
(or in a matched control group class) was 
measured by the student's final grade in that 
class. 

Implementation Measures 

On the post-test questionnaire, students in 
participating classes were asked to indicate 
how often their teacher had used the 
improvement-points method for scoring 
quizzes and tests and also how often the 
teacher had given out improvement awards: 

This year, some teachers at this school 
awarded improvement points to students on 
their quizzes, tests, or assignments. 
Students could win 10, 20, or 30 
improvement points depending on how 
much they were able to beat their "base 
score." How often did the teacher 
give improvement points on quizzes 
and tests in this class? 

Never Very Often 

1 2 3 4 5 

How often did the teacher give out special 
improvement awards (such as certificates, 
buttons, pencils, or bumper stickers) to 
students who were showing good progress? 

Never Very Often 

1 2 3 4 5 



For both items, the mean response of the 
students in each class was computed. In 
classes where the mean response to both 
items was 3.5 or higher, teachers were 
judged to have successfully met the 
implementation standards for Year 1. These 



standards were met by 11% of the 
participating classes; the data analyses 
reported here focus just on these classes. 
Similar (but somewhat weaker) effects are 
found in analyses that include all participating 
classes. 



Results 



Analysis Plan. The conventional way of 
evaluating program effects on continuous 
outcome variables after controlling for 
differences in pretest status is to conduct an 
ANCOVA at the student level, thus ignoring 
the fact that students are nested within 
classrooms. 

The recent development of hierarchical linear 
modeling(HLM) techniques makes it possible 
to estimate an ANCOVA-like model without 
erroneously assuming independent responses 
within classes (Bryk & Raudenbush, 1992). 
This type of HLM model (a random-intercept 
model with student-level covariates) takes 
into account the dependence among 
responses within classrooms and provides 
efficient estimates of program effects in 
unbalanced, nested designs. For analyses 
with continuous dependent variables, we 
used random intercept models with covariates 
to test for program effects. 



grade across all courses and all reporting 
periods) that students obtained on their report 
cards in the year prior to the beginning of the 
Incentives for Improvement Program. The 
average grades obtained by both experimental 
and control group subjects were in the 
satisfactory range. Experimental group 
students had nonsignificantly lower general 
averages than did comparison group subjects 
in the year prior to the intervention. 

Post-test grades. The second panel in 
Table I shows the unadjusted fourth-quarter 
grades obtained by students in the target 
courses at the end of the pilot year of the 
Incentives for Improvement Program. 
Students in the Incentives for Improvement 
program achieved significantly higher fourth- 
quarter grades in target courses than did 
comparison group students (random intercept 
model with no covariates, y = 5.1, g [one 
tail] £.025, ES = +.53.) 



One of our dependent variables-whether 
or not the student received a passing final 
grade- was a dichotomous, qualitative 
measure. Program effects on this variable 
were evaluated using logistic regression. 

Effects of the Incentives for 
Improvement Program on Students 1 
Report Card Grades. Report cards in 
Baltimore middle schools contain number 
grades rather than letter grades. Grades 
ranging from 90-100 are labeled "excellent," 
grades between 80-89 are labeled "good," 
grades between 70-79 arc labeled 
"satisfactory," and grades under 70 signify 
"failure." 

Pretest grades. The first panel in Table 1 
shows the general average (i.e., average 



However, this comparison between 
unadjusted fourth-quarter grades does not 
take into account each student's own prior 
general average. A more precise comparison 
can be made using m adjusted fourth-quarter 
grade for each child which indicates how 
much better or (worse) that child did (in the 
fourth quarter of the target course) than 
predicted based on his or her general average 
from the previous year. The average 
(adjusted) fourth-quarter grade is six points 
higher for experimental than for control 
students (random intercept model with one 
covariate, y * 6.4, g [one tail] £ .005, ES = 
+.66). The Incentives for Improvement 
Program is having a substantial positive 
impact on students' level of performance in 
participating courses; students in participating 
classes earn adjusted final grades that are 
almost two-thirds of a standard deviation 
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higher than students in matched non- 
participating classes. 



Insert Table 1 



Probability of Passing. By increasing 
student effort and improving performance, 
the Incentives for Improvement Program 
should increase the proportion of students 
who receive a passing grade. To test the 
effect of the program on students' probability 
of passing, logistic regression analyses were 
used to estimate the difference in probability 
of passing in experimental versus control 
classrooms after controlling for student's 
general average grade from the previous year. 
The Incentives for Imp r ovem ent program had 
a significant positive effect on students' 
probability of passing, B = .72, g [one tail] £ 
.01. Figure 4 shows the predicted 
probability of passing for students who had 
different levels of past performance. Note 
that the Incentives for Improvement program 
especially benefited students who were most 
at risk (those with low general averages from 
the previous year); 12% more of these 
students passed in experimental than in 
control classes. 



Insert Figure 4 



Effects of the Incentives for 
Improvement Program on Students' 
Self-Reported Effort, Intrinsic 
Valuing of the Subject Matter, and 
Self-Concept of Ability. In the 
Incentives for Improvement Program, 
students are assisted specific challenging 
goals and earn awards for attaining them. 
According to goal-setting theory (Locke & 
Latham, 1990a), these goals and incentives 
should lead students to work and study 
harder than students in nonparticipating 
classes. Panel 1 of Table 2 indicates that, 
holding pre-intervention effort constant, 
students in participating classes reported 
expending more effort (studying harder for 
quizzes and tests, working closer to their 



potential, etc.) than did students in control 
classes. 1 - 



Insert Table 2 



The Incentives for Improvement Program 
uses modest extrinsic rewards to recognize 
students for specific accomplishments. 
Convergent evidence from several studies 
(e.g., Deci, 1975; Kruglanski, Friedman, & 
Zevi, 1971; Lepper, Greene, & Nisbett, 
1973) indicates that the inappropriate use of 
extrinsic rewards can have detrimental effects 
on students' intrinsic motivation. But, as 
Lepper and Hodell (1989, p. 78) have 
argued, when rewards are given based on 
task performance and convey to children clear 
positive information about their increasing 
competence at an activity, the rewards are 
unlikely to undermine intrinsic interest. 

In fact, given the relatively low interest value 
of many school tasks, the recognition and 
reward structures in the Incentives for 
improvement Program could even enhance 
students' interest and enthusiasm because 
these structures ensure that every student is 
provided with an appropriate level of 
challenge. Throughout the year, each student 
in the program is asked to shoot for 
personalized goals that are neither trivially 
simple nor impossibly hard. The challenge 
of meeting such goals can be highly 
intrinsically motivating (Csikszentmihalyi & 
Nakamura,1989; Lepper & Hodell, 1989). 

Panel 2 of Table 2 indicates that the 
Incentives for Improvement Program had a 
marginally significant positive effect (of 
almost one-fifth of a standard deviation) on 
students' perceptions of the intrinsic value of 
the subject matter. 

It is well-established that when individuals 
succeed in attaining specific goals their ability 



1 - Adofesceau' ability perception, have a fitting impact on 
their effort and intrinsic valuing of the subject matter in junior 
and senior high school cornea (Mac Iver. Stipe* &. Daniels. 
1991). Therefore, program effects on efforts Mtd intrinsic 
vetoing of the subject matter went evaluated after controlling for 
students' pre -intervention ability perceptions. 



perceptions are increased (e.g., Bandura 
1986; Earley, 1986; Locke, Frederick, Lee, 
& Bobko, 1984; Mossholder, 1980). On the 
other hand, individuals with specific goals 
who do not reach those goals may develop 
ability perceptions that are less than or equal 
to those of lower-performing individuals with 
vague goals because the latter tend to give 
themselves the benefit of the doubt when 
evaluating their performance (Mossholder, 
1980). 



Thus, it was unclear whether program 
participants, who over time topically 
experience both some success and some 
failure in meeting the specific improvement 
goals, would develop higher ability 
perceptions than control group participants 
who may have only vsgue goals for their 
performance. Panel 3 of Table 2 shows that 
the program had a positive but marginally 
significant impact on students' ability 
perceptions. 



Discussion 



The results of this study show the substantial 
impact of individualized, improvement- 
oriented reward and recognition structures on 
students' grades, and thus on their 
probability of passing. The program's 
effectiveness is probably due to a number of 
factors. All students have a realistic chance 
of beating their individualized base scores 
with effort Even small improvements do not 
go unrecognized. In addition, by giving 
students a proximal, concrete goal to strive 
for on every quiz, the program may motivate 
more students to perform up to their potential 
on these quizzes, with beneficial effects on 
their grades. 

Finally, because teachers closely monitor 
student progress in a regular and systematic 
way, they become aware of the 
accomplishments of low-performing 
students. Teachers now see the small but 
significant gains that these students are 
making, and may be more likely than before 
to pass these students. 

Consistent with the predictions of goal- 
setting theory, there was a significant but 
modest positive impact of the program on 
students' self-reported effort. The 
challenging but reachable goals assigned to 
students in the program apparently motivate 
students to increase the duration and intensity 
of their learning efforts. 

Our findings show that the Incentives for 
Improvement program, in its pilot year, is 
accomplishing two major objectives in 



seeking to improve the academic performance 
of educationally disadvantaged students. 

First, it is providing an evaluative process in 
which educationally disadvantaged students 
share increased opportunities to experience 
genuine success in a challenging curriculum 
by earning recognition for academic 
improvement and by building upon this 
improvement to earn better grades and higher 
passing rates. The current press for the 
institution of higher and even "world-class" 
standards in our schools will be 
counterproductive if it increases the 
likelihood that the best efforts of 
educationally disadvantaged students will go 
unrecognized and unrewarded just because 
these students are starting out so far behind. 
On the other hand, if the establishment of 
higher standards is accompanied by the 
adoption of evaluation and recognition 
structures that provide students with specific 
improvement goals and regular improvement- 
focused feedback and recognition, then 
educationally disadvantaged students may 
have the impetus and support they need to 
actually reach these higher standards over 
time. The Incentives for Improvement 
program is a step in this direction. 

Second, teachers' expectations that students 
will succeed academically - that all students 
can learn - are a vital part of motivating and 
effectively teaching currently low-achieving 
students. Teachers are constantly advised 
that they must have high expectations for all 
students and mi st offer all students a high- 
content curriculum. Unfortunately, when 
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teachers take this advice, current evaluation 
practices in schools almost pre-ordain the 
failure of students who start out far behind in 
their learning. These failures reinforce 
teacher expectations that educationally- 
disadvantaged students are incapable of 
mastering advanced-level knowledge and 
increases the pressure on teachers to return to 
a watered-down curriculum. 

The Incentives for Improvement program 
changes this picture dramatically by giving 
educational ly-disadvantaged students a 
genuine opportunity for success in a high- 
content curriculum and by providing teachers 
with direct evidence weekly in their own 
classrooms that their low-achieving students 
can indeed improve their academic 
performance. Thus, teachers' expectations 
that all students can learn, in the Incentives 
for Improvement classroom, are reinforced 
and increased by concrete evidence of 
academic improvement. 

A Helpful Innovation or a "Bad Idea 
Whose Time Has Come?" Finn (1991, 
pp. 109-1 11, 122 224) has called the practice 
of giving low-achieving children "positive 
reinforcement and favorable feedback in the 
form of encouraging teacher comments, 
upbeat report cards" and so on, "a bad idea 
whose time has come." He sees this practice 
as partly responsible for parents' and 
youngsters' complacency and for their 
tendency to overrate the youngsters' 
accomplishments. Finn blames colleges of 
education and education research institutes 
for promoting the use of undemanding 
standards by encouraging the use of 
evaluation systems that put success within the 
reach of all students. . 

When calls to implement a more responsive 
evaluation system are misunderstood as calls 
to avoid giving students any negative 
feedback, these calls are indeed 
counterproductive. Any evaluation system 
that rewards low-achieving students 
regardless of whether they are showing 
progress deserves to be criticized as 
something that is more damaging than 
helpfuL On the other hand, it is a grave 
mistake "to blacken with the same brush" 
rigorous progress-oriented evaluation 



systems such as the Incentives with 
Improvement program. In these systems, 
success and its rewards are indeed accessible 
to all students, but only if all students work 
harder and smarter and thus raise their level 
of performance. 

In the Incentives for Improvement program, 
students receive realistic information 
concerning their current level of performance 
frequently, including clear negative feedback 
when they fail to reach their assigned 
improvement goals. Decades of research on 
student and worker motivation make it clear 
that it is possible, indeed essential, to 
implement such evaluation systems that are 
both responsive (every individual has a 
realistic chance to achieve success) and 
challenging (none are likely to succeed unless 
they work up to their potential). 

A recent trend in educational assessment is to 
replace traditional quizzes and tests with 
"authentic assessments" or "true tests" that 
ask students to demonstrate their mastery 
while showing "that they can produce 
something of value to themselves and others 
- an argument, a report, a plan, an answer or 
solution; a story, a poem, a drawing, a 
sculpture, or a performance; that they can 
conduct an experiment or deliver a persuasive 
oral presentation" (California Assessment 
Program, not dated). Does the Incentives for 
Improvement program require teachers to 
continue using traditional quizzes and tests? 
By no means. Several of the participating 
teachers used richer, more active and realistic 
types of assessments fairly often. In their 
feedback to students, these teachers gave 
each student - in addition to their comments 
and ratings of the student's performance on 
specific dimensions (e.g. organization, 
content, creativity, handling of questions) -- 
a summary score (e.g., "summing across all 
dimensions graded, you received 80% of the 
total possible points"). These summary 
scores were used for computing improvement 
points, figuring new base scores, and 
determining awards. 

One caveat is in order. As assessment tasks 
become more complex, increased 
achievement will be associated with increased 
effort and persistence only for individuals 
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who know effective task strategies (Locke & 
Latham, 1990). For example, an 
improvement-oriented evaluation system may 
prompt students to work hard on an 
"authentic" task, but this effort is unlikely to 
pay off unless students have also received 



effective instruction in how to approach such 
a task — for example, direct instruction in 
helpful, domain-specific metacognitive and 
problem-solving strategies (Nickerson, 
1988). 
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o Flpjuref . Improvement points earned by fictitious students on the first quiz in Round 1. 
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Onure 2 , New base scores and average Improvement points earned by students during Round 1. 



Base Score Sheet: Teacher ALUs 



Period 3r, 





1 

Scot* 

1 


Scora 

2 


Ham 

Scora 


Scora 

A 

m 


Scon 

9 


Scora 
c 

w 


A AAA 
o#0O 

Scora 

7 


Aa aa 

Score 

■ 


■ami 

Scora 
9 


Scora 
10 




to 


6 3 


65" 












, 




Rock* Q . 




1 


















6r;nh+w\ k F. 


lo 


7 V 


9 T 


















£o 


*- 


















— - — — . ..i 


ST 




















— r ^ 


GS 






















If 


go 


















^ 


7o 


74 


to 


















Ho 




1 Q 
















— — * y >y — - 


7o 


So 




















5*0 


Co 




















£5" 


SI 














. — . 





Tryi ttf J. 


£5* 


18 

— IK 














i 


— — _ 




£c? 


9s 




















6o 
















. 














































4 




















^ 




3 























ERIC 



R fl ure 3 - Record of students' base scores up to the start of Round 3. 



Figure 4 

Program Effects on Students' Probability of Passing 
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Table 1 



Report Card Grades: Means, V (HLM) Eatimatea of Program Effects, and Effect 
Sites IN - 28 claaaea) 



Panel 1: 



Mean "General Average" on Report Card 
in Year Prior to the Intervention 



y (HLM) 



Effect 
Size 



Experimental 
74 



Control 
76 



-1.6 



-.27 



panel 2: 

Mean Unadjuated Final Grade 
in Intervention Year 



7 {HLM) 



Effect 
Size 



Experimental 
79 



Control 
74 



4-5.1*** 



+ .53 



ffanETJl 



Mean Adjusted Final Grade 
in Intervention Year* 



y (HLM) 



Effect 
Size 



Experimental 
78 



Control 
72 



+6.4**** 



+ .66 



Controlling for differences in students' general averages from the prior 
year. 



*E <_.10 **£ < .05 ***£ < .025 ****£ < .005 (one-tailed testa) 
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Table 2 

Students' Self -Reported Effort, intrinsic Value of Subject Matter, and Self- 



Concept of Ability: Adjusted Means, V (HLM) Estimates of Program Effects, and 
Effect Sizes (ZV - 32 classes) 



Panel 1: 

Adjusted. Mean Fourth-Quarter Eff ort 8 



Experimental students Control students 
5.51 5.28 



Y (HLM) 



,23** 



Effect 
Size 



+ .20 



Panel 2: 

Adjusted Mean Fourth-Quarter Perceptions of 

the Intrinsic Value of the S ubject Matter 13 f (HLM) 



Experimental students Control students 

5.31 5.06 .25' 



Effect 
Size 



+ .18 



Panel 3: 

Adjusted Mean Fourth-Quarter Self-Concept 
.of Ability 0 



E. ~«/. omental students control students 
5. HO 5.24 



Y (HLM) 



16* 



Effect 
Size 



+ .13 



a Contr?lling for pre-test differences in effort and self-concept of ability. 

^Controlling for pre-test differences in perceptions of intrinsic value and 
self -concept of ability. 

Controlling for pre-test differences in self -concept of ability. 
*£ < .10 **£ < .05 (one-tailed tests) 



Appendix 



Questionnaire Items Used to Measure Students' Perceptions 

Student's Perception of the Intrinsic Value of the Subject Matter 

How excited are you to leam about this subject matter? (not at all excited)... (very excited) 

How much do you enjoy learning about this subject? (not much at all)... (very mucU) 

How much do you care about learning a lot about this subject? (don't care a* all)... (care very 
much) 

« 

How much do you like working on the assignments in this class? {not at all). ..(very much) 

Do you do things for fun outside of class that are related to or have something to do with what you 
are learning about in this class? (never). ..(yes, a lot) 

Student's Self-Concept of Ability 

How good are you in this subject? (not good at all)... (very good) 

How good do you think you are in this subject compared to other students in the class? (much 
worse than other students). ..(much better than other students) 

How often do you feel smart in this class? (never). ..(very often) 

How much natural ability do you have in this subject? (no ability at all)... (a lot of ability) 

Student's Effort 

If a student works to his or her highest potential in a class, then we could say that he or she is 
putting forth 100% effort to learn the subject matter. How much effort do you usually put 
forth in this class? (0% - / am not trying at al[)...(10Q% - lam working to my highest 
potential) 

How hard are you working to leam about this subject? (not hard at all)... (as hard as I can) 

How hard do you study for tests in this class? (Just enough to pass). ..(whatever it takes to get a 
good grade) 

How hard do you work in this class? (much less than most classes). ..(much more than most 
classes) 
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